Section EDUCATION AND EDUCATIONAL RESEARCH 


APPLICATION OF ADAPTIVE COMPUTER TESTING AS A 
MEANS OF INTERACTIVE LEARNING FOR BUILDING AN 
INDIVIDUAL EDUCATIONAL TRAJECTORY OF A 
STUDENT 


Assoc. Prof. Eliseeva Dina Yu.! 

Dr. Fedosov A.Yu.” 

Assoc. Prof. Mnatsakanyan Olga L.? 

Assoc. Prof. Grigoreva Svetlana V.4 

Assoc. Prof. Dmitrieva Tatiana V.° 

'.3.4 Russian State Social University, Moscow City pedagogical University, 
Russia 

Russian State Social University, Russia 

> Russian State Social University, Finance University, Russia 


ABSTRACT 


Building the process of education effectively and qualitatively taking into 
account educational needs and students’ individual characteristics is possible by 
developing of the special technics of education, in terms of which a student not only 
will repeat the actions of a teacher but first of all will use different interactive and 
adaptive educational technologies. 


The modern computer technologies because of high crucible carrier size and 
high processing speed allow to enlarge the borders of materials used in an 
educational process because of highly developed ways of analysis of a dialogue 
with a user and considerably adopt transfer of knowledge to a concrete learner. 


The teaching technique of computer testing suggests realization of new 
approaches to a building of teaching courses by special organization of information 
block of a discipline in the form of sections and modules, structuring of test 
materials due to the module division in accordance with information structure of 
learning material, division of module testing material into the groups of test 
questions and tasks in keeping with the content and arrangement of groups in order 
of difficulty of testing materials, appliance of consistent approach allowing to 
realize several sorts of adaptiveness while testing students’ knowledge. 


We offered the consistent approach, allowing to realize practically several types 
of adaptiveness while estimating knowledge by computer testing. 


1. The changing of strategy of examination depending on the student’s 
answers, legalized by means of special structure of examining algorithm. 

2. Testing on assignment, which is divided into the training, realized 
independently for the estimation of knowledge by a student himself, and control, 
realized under the supervision of a teacher for the objective estimation of real 
knowledge, and progress check and final check are important and particular case. 

3. Testing on profoundness of the examination — to the examination level. 

4. The testing by volume of the checked material subdivided depending on 
material volume (the module, the section or several sections (progress check), all 
discipline (final check). 
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The given possibilities are provided with the organization of the information 
and testing material for subsections of a course and are considered in the testing 
algorithms. 

The offered approach maximally takes into account essential characteristics 
of computer process of an examination and allows to build an individual path of 
training students by using various interactive means. 

Keywords: interactive teaching technics, adaptation, computer testing, 
testing algorithm 


INTRODUCTION 


The using of adaptive computer testing as a way of interactive learning for 
building an individual educational trajectory of a student. There is a principle of 
comprehensive registration the individual features of the students and the fullest 
satisfaction of their educational needs is based on the construction of a modern 
effective educational process. The realization of such learning process is 
possible thanks to the way of introduction of the special methods based on using 
the interactive and adaptive learning technologies. 


The modern information technologies make it possible to use resources 
which linked with the educational processes, and widen the possibilities of 
materials used in the learning process, also due to the fact that developed 
methods of analyzing the dialogue with the user, and substantially give the 
opportunity to adapt the transfer of individual knowledge to a particular student 


[2]. 


A wide using of testing in Western countries associated with the definition 
of professional suitability and skill level was due to a number of reasons [5], 


[6], [7]: 


. a business interest to the maximum effective using of human resources; 

° the aspiration to reward a talent regardless of its social background; 

. the need of Americans in national standards, the desire of society to 
maximize the return on each of his abilities and the availability of methods 
to identify these abilities. 


The classical testology of foreign countries is based on the following basic 
postulates: 


° human abilities are inborn and therefore, practically, unchanged (exactly 
this statement led to a comparison of abilities with the level of knowledge); 

. a high level of abilities is rare and the ability of people is distributed in 
accordance with the law of Gauss; 

. ignoring the qualitative development of a person - both mental and 
psychical, understanding the differences in the abilities of people to 
quantitative indicators; 

. the principle "decided - did not decide" namely control is subject to only 
to the final result of the intellectual activity when task execution is not 
diagnosed and not considered [4]. 
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Of course, there is a modern trend, which puts in doubt the basic approaches to 
testing, there is a rethinking of the role of education in the development of human 


abilities; 


For the period from the early 90s of the 20th century to the present time, there 
has been a branch of science in Russia - the theory of pedagogical measurements, 
which has its own methodology, methods, instrument for developing and 
implementing qualitative pedagogical tests, magazines, innovative works, 
collections of materials on test subjects are published. 


Modern Russian testology develops in several directions: 


theoretical and methodological foundations of the testology are considered 
by such scientists as V.S. Avanesov, A.V. Abramova, M.S. Bershtein, M.I. 
Pobedovy and the others; 

a place of testology in the system of pedagogical sciences - B.P. Bittinas, 
L.I. Nataeva, I.I. Tikhonov and the others; 

the questions of the typology of test tasks are studied by V.S. Avanesov, 
V.P. Bespalko, D.S. Gorbatov and the others; 

the features of psychological testing are studied by A. Anastasi, G.S. 
Kovaleva, V. Kline and the others. 

Computers have been widely used in testing since the beginning of the 21st 
century. There is a whole direction in pedagogical research - computer 
testing, which has a number of undeniable advantages over traditional 
testing, namely [3]: 

it’s an opportunity to use more different types of visibility (audio and video 
fragments, diagrams, drawings, computer models, etc.); 

the computer more than the presence of the experimenter, motivate the 
examinee on its own, and at the end the diagnostic value of the results are 
higher; 

the influence of the personality features of the experimenter and random 
behavioral factors are reduced; 

it becomes possible to compare the experimental facts, obtained by 
different researchers; 

any complicated information is automated, it is possible to get quickly 
comparable results from large block of information, to save and compare 
it’s; 

it is enough easy to combine the functions of diagnostics and correction of 
knowledge in training systems; 

there is an opportunity to use such forms as imitation exercises, the 
analysis of situation , action tests which are associated with simulators; 
the opportunity to include the psychodiagnostics elements, attitudes, 
feelings, impressions, influence of cognitive styles in the testing sphere; 
the opportunity to use an adaptive testing strategy when the research 
strategy changes depending on the results obtained previously. 


Besides this, computer testing allows you to save material resources, which 
usually are spent on the production of blank tests. With its help, you can improve 
information security by protecting electronic files, increase the range of measured 
skills and abilities, using a variety of computer capabilities - interactivity, 
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multimedia, etc. Students are usually have a positive attitude to computer 
testing, noting the convenience of quickly obtaining test results and the absence 
of various forms that need to be filled, it is enough to choose the answer with 
the mouse button, the innovative nature of control, a more accurate mark of 
knowledge and skills. But, at the same time, it is indicated a number of 
inconveniences, for example, that often there is a fixed order of the task, strictly 
limited time for each task after which the computer itself passes to the next task, 
regardless of the testee's desire, sometimes it is not possible to look through the 
test at the beginning of work with it, change the answers to the previous task, 
pass the task, and also there are difficulties in writing mathematical calculations. 


It should be attributed to attention the fact in computer testing, that it’s 
necessary to account for the different level of computer skills among the 
students, because it influences on the validity of the test results. It’s usually 
practice to include training exercises, instructions, in advance introduce students 
to the interface, do the repetition testing to reduce the influence of this indicator 
on the test results. 


Hardware or computer tests have a number of significant differences from 
the traditional tests, in particular they can be classified by the degree of 
automation for: 


. automated - are held by the teacher using computer facilities 
. automatic - are held using computer facilities 


Computer testing can be in various forms. 


1. The test is entered in a special sheath that allows you to display the task 
on the computer screen, evaluate the results of the task, generate a table of 
test results, process the results. 

2. Testing with the help of automated generation of variants of the test 
options, which is done with the using of tools. The variants of tests are 
formed from a bank of calibrated test tasks before testing or directly 
during the testing procedure. 

3. Computer adaptive testing is based on the principle that the student does 
not have the sense to give tasks that he will surely do without any 
difficulties or vice versa, he will not do because a high degree of difficulty, 
so the computer chooses a test task depending on the previous answer of 
the person who solves its test. 


Test tasks that use the possibilities of computer testing are the most 
perspective direction in the development of pedagogical measurements because 
of the increase in the content validity and the great potential of such tasks for 
increasing the informativeness of pedagogical measurements. Using tests tasks, 
it is possible to much wide the range of assessment of knowledge, skills, and 
various indicators of mental activity that cannot always reveal traditional blank 
testing: the level of analytical and synthetic activity of the student, the flexibility 
of the cogitative process, the speed of generalization of information, etc. 


It is possible to include visual information here when develop test tasks for 
computer testing - realistic nature (cinema, photo) and synthesized nature 
(animation, drawings), which introduces dynamics in the performance of the 


26 


Section EDUCATION AND EDUCATIONAL RESEARCH 


test, as well as audio information requiring a voice answer. Nevertheless, it is 
necessary to take into consideration the fact that the variety of sound and visual 
images during testing makes the students get tired, which requires a shortening of 
the length of test. 


It has a number of features and a system for calculating final marks. If the 
interactive and multimedia technologies were not included there the calculation is 
made in the traditional way - summing up the points for each task. In case if the test 
uses modern multimedia and interactive technologies, it leads to the need to assess 
a number of different skills - creative, communicative, and subject-specific, which 
requires polygamous evaluations. Such a verification requires a more complicated 
automation process, the need to "training" a computer program to evaluate any 
detailed answers. 


The creation of structure of knowledge is a separate problem, when the general 
concepts and objectives of the course are formulated in the early stage of training, 
and the objects are built in the next phases, the tasks of the discipline are studied. 
When the material is considered this structure is always followed, however, this 
condition is not always taken into consideration. A typical situation is a chaotic 
mixture of test tasks that refers to the different phases of learning. 


From the point of view of the creation a rational structure of knowledge, it is 
necessary not only to correctly define the structure of single tests, but also the 
structure of all the mass of test tasks. It is possible to divide large parts (sections) 
into parts for more convenient organization of computer testing of educational 
material that will differ from ordinary section. 


APPROACH 


We used the method of analysis of literature and Internet sources on 
philosophical, social and psychological issues such testology and computer testing, 
as well as the study and synthesis of psychological and educational experience; 
pedagogical experiments to approbation of methodology of adaptive computer 
testing. 


BUILDING AN INDIVIDUAL EDUCATIONAL TRAJECTORY 
OF A STUDENT ON THE BASIS OF APPLYING THE ADAPTIVE 
COMPUTER TESTING 


The methodology of adaptive computer testing offers the realization a new 
methods to the design of educational disciplines thanks to the special organization 
of the information block of discipline in the form of sections and modules, the 
structuring of test materials based on modular breakdown in accordance with the 
information structure of the educational material, the separation of the modular test 
material into groups of test questions and tasks in accordance with their content and 
ordering of groups by the degree of complexity of test materials, using a single 
method which allows to realize the several types of adaptability in testing students’ 
knowledge. 


It was created the invisible way by the authors, which helps to realize in practice 
several types of adaptability in the diagnosis of students' knowledge by computer 
testing. 
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1. Adaptability, which is related to the interview strategy depending on the 
student's answer- it’s a traditional way of adaptability, it can be realized 
thanks to the construction of a special structure of the interview algorithm. 

2. Adaptability in depending on the purpose of testing - is divided into: 

a) educational, which is carried out individually to assess 
knowledge by the students themselves, 

b) control, which is carried out under the supervision of a teacher 
for an objective assessment of the real level of knowledge, 
midterm and final control are important special events. 

3. Adaptability which is associated with the depth of the interview (the level 
of knowledge testing) - is possible one-level, two-level and three-level - 
by the number of groups of test tasks. 

4. Adaptability which is associated with the structuring of the volume of the 
test’s material (at the level of a subsection (module), section or several 
sections and at the level of the material of the whole discipline) - these 
opportunities are provided by the organization of information and testing 
material on the sub-sections of the course and are taken into consideration 
in the testing algorithms. 


A method that has been offered takes into consideration the maximum level 
all essential characteristics of the computer knowledge verification process and 
allows you to build an individual trajectory of student learning using various 
interactive tools. 


If it’s possible, while you create a computer training systems, it is necessary 
not only to copy the actions of the teacher, but also to introduce new adaptive 
methods that are realized only with the help of computer systems. The most 
convenient form of single tests for computer knowledge testing are tasks with 
one correct answer. To construct an adaptive algorithm for testing knowledge, 
along with the approach adopted in the theory of modeling and parameterization 
of pedagogical tests (Item Response Theory), it is suggested to use the following 
ways: to present all testing process as a series of steps consisting of single test 
tasks logically connected to each other or independent from each other. At this 
stage, the all strategy is unchanged. The transition between the steps is based on 
the results of the answer to the tasks. Thus, at every step of the test, the 
automated system must do two main functions: 


1. the mark of the student's actions at this step (answers to single test 
questions) is an evaluation function, 

2. the transition to the next step (determining the number of tests on it, its 
content, etc.) is a control function. 


The verification of the knowledge of each individual student is necessary 
both for all forms of individual training and for group training. Examine the 
possibilities of adapting computer knowledge control for these types of 
verification. 
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For individual verification, it was offered to introduce a two-level adaptation of 
the knowledge control process: 


1. adaptive evaluation of the results of each single alternative test given 
to the student, 

2. adaptive construction of the entire testing process, taking into account 
the composition of test materials and the correctness of these 
responses. 


Consider the first level of adaptation. 


A general formula for estimating the results of a single test is proposed. It has a 
multiplicative structure that contains the R+ - basic component of the assessment, 
and R- - the penal component of the assessment. 


R=R({V}.N, {P}) =R+({V}, N) x R-(, {P}) (1) 
R - the final evaluation of the student's response to the test task, 

R+ ({V}, N) - the basic component of the assessment, the valuation function, 
R- (N, {P}) - the penalty component of the evaluation, which reduces it. 


{V} - information received from the student (one or more options for the correct 
answer), 


N is the number of answer options, 
{P} is a penalty function. 
We define the basic estimate R+ of the answer to the test task. 


We will understand by the basic evaluation of the answer the "pure" numerical 
characteristic of the answer, which estimates only its truth - proximity to the correct 
answer variant. 


The classical way (logical evaluation): 
0- if the answer is wrong (incorrect) 


R= 1 - if the answer is right (correct) 


The suggested way (real valuation): R €[0,1]. 


In the classical version, the answer “yes-no” is actually a logical value. It is 
more convenient for the teacher, but, at the same time, it is much more simplified 
than the material estimate and does not allow you to subtly assess the various 
nuances in the verification of knowledge. This shortcoming is intended to eliminate 
the proposed continuous real valuation. 


{ V} — the information received from the student (plurality of possible answers). 
The classical way. 
The plurality {V} must contain one answer. 


The offered way. 


29 


NORDSCI CONFERENCE 


The student can indicate several possible options for the correct answer {V}, 
which, in his opinion, include the correct answer to Vist. The transition to the 
multiple response allows more flexible take into account the level of knowledge, 
taking into account their temporary forgetting, the psychological factors of the 
examination situation. In this case, the basic component of the estimate R+ 
({V}, N) is suggested to be performed using the following relationship: 


0, if [V|=Oor Vist ¢{V}; 
R= (2) 
R({V)), Vist €{V}, [V|>0. 


The evaluation function R+ ({V}, N) in formula (2), means an estimate that 
is made when there is a correct answer in the group of |V]. 


Linear, quadratic and cubic dependences were investigated during 
constructing the function R (|V|). The best results are given by cubic 
dependence. It is represented by the formula (3): 


R*(X, N) =(N - V)[(-N3 + 9N2 - 21N + 15).(N - V) + (N2- 6N + 7)] / [2(N- 
1)2(N-2)?] (3) 


The verification of the formula for practically significant values of N= 3. /10 
(the number of variants of the answer) showed that the monotonicity of the 
estimation function is provided only at N = 3,4,5,6. However, the using of 
increased values of N for computer testing systems is irrational due to an 
increase in the total time spent on testing knowledge. As practice shows, the 
optimal value is N = 5. With it, on the one hand there is a small positive estimate 
with random guessing of the answers (0.2). On the other hand, the number of 
answer options is not too large and does not do the testing process long. At the 
modern level of information technology development, it is possible to 
effectively track only the r time of answer. To calculate the optimal response 
time of the Topt (N) was suggested to use the formula (4). 


Topt (N) _ Tanswer “F N.T question, 
(4) 
where T answer is the time for evaluating the correct answer to the test question, 
Tuestion - the time for evaluating one answer option. 


The values of Tanswer and Tguestion Should be determined, based on complexity 
of the questions. To construct a penalty function for {P} = T, it was offered: 


1. to accept the waiting time limit equal to three times the maximum response 
time; 

2. linear decrease of the penalty function R- (N, {P}) from | (where T= T opt 
(N)) to 0 (where T = 3T opt (N)); 

3. for T> 3T op (N), the quantity R- (N, {P}) = 0, ie. a very high time of 
waiting for a response about the use of unacceptable sources to respond to 
the test. 


In the analytical form, we have the piecewise linear dependence which is 
represented by the formula (5) 
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lifT<T.; 


opt ? 
R’(N,{P})= 4 0,5(3-T/T,,, ), if Topt < T < 3Topt 
0,if T > 3Topt. 


Thus, it was defined a number of formal features, according to which it is 
possible to effectively adapt a single alternative test and was given the 
corresponding mathematical model of the test, for which it is possible to implement 
a software implementation. 


Let’s consider the second level of adaptation [1]. 


The following structure of the main materials on the academic discipline was 
offered for practically implement various types of adaptability in the submission 
and control of knowledge: the informational block consists of 3 - 5 sections, and 
each section consists of 3 - 5 subsections - modules. As electronic training materials 
are always built on the basis of a literary presentation of the material, it were offered 
the following actions to highlight modules and sections: 


1. to take one paragraph as a module, which have an independent meaningful 
or several consecutive paragraphs containing a common meaningful 
content; 

2. one section is placed in the information area, in which the completed part 
of the studied discipline or several interdependent consecutive chapter is 
fully revealed. 


The testing block is consists of structured test tasks for all course modules. Each 
module, in accordance with part of its content, is suggested to highlight the 
following groups of test questions and tasks that for each specific subsection should 
be arranged in order of increasing complexity: basic interview, theoretical questions 
of a high complexity, standard tasks for the material passed. 


It was offered a unification of content and test materials which gives as 
opportunity to break the study of the material and effective control of its 
assimilation into separate finished parts. Each group of testing materials is divided 
and contains a set of 20-30 questions or tasks. This method of the structuring test’s 
materials allows in a computerized version to implement a step-by-step increase in 
the complexity of verification tasks. These opportunities are provided by the 
organization of information and testing material on the sub-sections of the course 
and are taken into account in the testing algorithms. 


As the generating polling tree that realized the polling algorithm with the 
minimum number of single tests, we will take the structure which is shown in Figure 
1. 


The generating tree is built on the basis of rules. 


If there are all the basic answers are incorrect, the final evaluation is 0 points 
(the leftmost top in the upper tier), the poll stops. 
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Picture 1. The structure of the generating tree for model testing of the first 
(a), the second (b), and the third (c) levels 


2. If there are all the basic answers are correct, the final evaluation is 100 
points (the extreme right top in the upper tier), the poll stops. 


3. If there are one correct answer in the base phase (the middle top in the 
upper tier), the intermediate evaluation is 50 points, the poll continues in the 
specifying phase (the lower tier). 


4. There is anumber of questions reduced by half compared to the base phase 
in the specifying phase. 


5. If there is an incorrect answer was given to the specifying question, the 
number of points are the same as in the intermediate phase - 50 (the extreme left 
top in the lower tier). 


6. If there is the correct answer was given to the specifying question, the 
number of points is increased to 70 (as a result, the percentage number of correct 
answers is 


100 * (2/3) 70) - (the extreme right top in the lower tier). 


It was suggested to use two extra rules, when you build the algorithm trees 
for two- and three-level model testing. 


1. Entering the distribution of the initial 100 points according to the levels of 
testing, taking into account their value. 

2. Itis necessary to use the next control function at the each level: 

. in case when the estimate given before does not exceed the half of the 
maximum possible estimate, there is a transition to the specifying phase of 
testing with the next level of finishing test 

. otherwise there is the transition to the base phase of the next level with the 
continue of the testing process. 


The evaluation and control functions of the unit testing algorithm were 
described before in a formal form, and was graphically presented as trees for the 
situation of a classic single test. The formalization of both testing functions 
allows to use it in case of an adaptive single test and create an suitable algorithm. 
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Examine its verbal definition. 
The initial data: 
1) t- it’s the scaling coefficient, 
2) N - is the number of test’s levels in the module, 
3) P [1: N] - the allocation of points by levels. 


The formulation of the problem. It is necessary to conduct testing and obtain a 
final score T, which assesses the level of knowledge of the student. 


The following algorithm was made to solve the problem. 

The algorithm for modular adaptive testing. 

Step 1. Initial assignments: 7: = 0; 

Step 2. Arithmetic cycle by test levels, variable i varies from 1 to Nu. 


Step. 2.1. Testing at level i (2t single tests), the result of which is the 
determination of the real number of correct answers responses and their score B jp. 


Step 2.2. Testing the effectiveness of testing: Bi, < 0,5 Pi. 
If the condition does not hold go to Step 2.3, otherwise on Step 2.4. 


Step 2.3. Clarification of knowledge at the actual level i — implementation t 
single tests. As a result, a real number of correct answers riu and additional points 
are determined: Biu =0,25 rj, (Pi /t). 


Step 2.4. The determination of full assessment at level i by formula (2.21): 
B; = Biy + Biv, if i had done both form, 

Bi = By, if only basic testing has been done. 

Step 2.5. Increasing the total score: 7:=T+ Bi. 


Step 2.6. If clarifying testing has been done, the procedure is completed, 
otherwise-the transition to the next level of testing. 


Step 3. The receiving the test’s result: total score T. 


The developed algorithm has no complicated structure, it is simple in software 
implementation. 


The considered model of a single alternative test raises the degree of adaptability 
in comparison with the limit classical test due to a more flexible evaluation of 
response options and consideration of the time parameters of the response. To 
realize adaptivity in the proposed model in the extended version use the capabilities 
of the computing system. It allows to realize more fully an individual attitude in 
computer training at the first stage of knowledge testing — at the level of a single 
test. The suggested variant takes into attention to the maximum the essential 
characteristics of the computer process of knowledge testing. 


When the individual testing of students will be done in the whole group the 
teacher must give the evaluation based on the scores received during testing, and in 
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the case if there are students who not so good in studying, make a decision on 
the group correction of their knowledge level. 


Group forms of studies in the universities are supplemented by the individual 
consultations, which have an auxiliary character in relation to group classes. The 
main types of correction in practice are: 


1. individual study using the sources of information recommended 
by the teacher, 

2. the consultation in addition to the main training lessons 
(individual or group), 

3. additional explanation of the material during lessons (group), 

4. studying the educational material on short-term courses (group). 


The first variant of corrective actions is the independent study by the student 
the material which was missed by different reasons and imply primary (in case 
of missing lessons) or deep study of missed topics in the main extra and 
educational literature (in case of special interest on the part of the student). It 
gives a positive effect, mainly, only for students with a tendency for independent 
study. 


The second type of correction basically use when the student has not 
understood any particular issues of the studied material during the class. Then 
he can ask clarifying questions to the teacher in specially extra consultation 
hours or during the break between lessons. This type of correction is made for 
an individual type of knowledge correction. 


The third type of corrective actions is a shortened version of the material, 
taking into attention the results of testing. It expected the teacher, studying the 
results of group testing, makes the conclusion that it is necessary to repeat some 
part of the same passed material in short time, emphasis the students’ attention 
on a certain basic moment and for this purpose takes some extra time during one 
of the lesson. The method is effective, because in the process of preparation and 
testing itself, students already on a semantic level learn the material being 
studied, and the teacher's second explanation is more deeper for them. 


The fourth type of correction is used in case when there is a group of students 
who for one or an another reason have not studied a significant part of the 
material (pass, illness, etc.). Itis economically good to use this type of correction 
in the group variant, because such lessons give extra pay hours for teacher’s 
time, reservation of the audit fund, etc. 


The definition of the optimal type of correction for all students in group is a 
very complicated problem even for an experienced teacher. With the automated 
analysis of the results of computer testing, the corresponding computer program 
lacks the skills and intuition of the teacher. It should be changed sufficiently by 
the corresponding calculation algorithm, in which all the basic steps are 
presented in a formalized form. 
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We will use the following designations to do more formal writing of the 
algorithm: 


- R — the total score of the final control given in the university, 


- A - the permissible part of the total score when the student is allowed to the 
final certification, 


- D—part where R is in the final evaluation. 


After computer testing in the group for the specified sections of the course, the 
analyzing algorithm must determine: 


1. sub-group with low achievers, 
2. to forma correction group, 
3. determine the best ways to correct knowledge in this group. 


The algorithm for group testing analysis and determination of optimal corrective 
actions. 


Initial information: 


1. points B= {B', B’,..., BY} for a group of k students received during 
testing educational material, 

2. the attendance and part of attended lessons q= {1, 92. gk} by 
students during the period of studying the tested material. 

3. the minimal part of the attended classes [g], which necessary for the 
quality understanding of the tested material. 


The formulation of the problem. It’s necessary to: 1) define a subgroup with 
reduced student’s achievement, 2) form correction groups, 3) define the best ways 
to correct knowledge. 


Step 1. Exception pseudo-zero assessment. After checking the condition B < 
100(1/N) with N=5 and dropping students with these estimates, we get a new 
adjusted group of s students (s <4) and their test results B’ = {B"’, B’’,..., B’ }. There 
are all estimates exceed the value of 20. 


Step 2. Statistical analysis of the corrected group. The calculation of the center 
of gravity C’, dispersion D' and SKO S" for the corrected group by the set of its test 
results B’. 


Step 3. The formation of a subgroup with reduced academic progress. We 
include all students who have the results B’’’ and satisfy the double inequality: C’ - 
15s < BNSC +05S". 


As a result, we get a characteristic subgroup with a reduced level of testing 
results. The number of students in it will be denoted by r. 
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Step 4. The definition of the corrective score 4 for a subgroup with reduced 
academic progress: 


= d, npuld| <5; 
G2 C M1 AS Az puld| 
5—unaye. 


Step 5. The calculation of modified scores for a subgroup with reduced 
academic progress: Forj = 1, ...,r 


If d>0, to B/'"= B"'+/, otherwise B= Bi'-/ . 
We also correct the lower and upper limits of the PPU. 


Step 6. The formation and analysis of the correction group (mark less than 
A). It consists of two subgroups: 


a) students whose test’s marks are less than the lower limit of the subgroup 
with reduced academic progress(C’-1,55S’) (pseudo-zero estimates and 
estimates lying in the range of 20 points to the lower limit of the PPU). 

b) students from the PPU group, whose marks is lower than the permissible 
A: 

BIT SA, 


Step 7. The correction of knowledge in subgroup a). The whole subgroup is 
sent to courses because of its low level of knowledge. 


Step 8. The correction of knowledge in subgroup b). The total number of 
students in subgroup b) is denoted by p. It is necessary to find out in what form 
it is rational to carry out group correction. For each student of the subgroup, the 
condition of sufficient presence verify: gi = [gq]. Denote the part of the subgroup 
for which this condition is not satisfied by pn. The following rule for choosing 
the optimal form of knowledge correction is offered. 


Step 8.1. If pn < 0,3p (who missed the lessons really rarely, but they have a 
level of achievement close to the minimum acceptable), use only the form 3 
(additional group explanation of the material during the main lessons) in this 
group in order to minimize the value of additional courses. 


Step 8.2. If 0,3p < pn < 0,7p (there are a lot of students who missed the 
lessons, but they do not in the whole subgroup b)), then both forms 3 and 4 
should be used together. In this case, there is a large group of students with a 
large number of absences they attend courses together with those who got 
pseudo-zero and close marks. For students who did not understand training 
material rather good during a stable attendance of lessons, it’s enough to explain 
the main training course. 


Step 8.3. If pn > 0,7p (students who missed many lessons compose almost 
the entire subgroup b)), then use only Form 4 (additional courses), as students 
who attended lessons basically passed the tests successfully. 
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THE COMPLETION OF THE ALGORITHM 
Suggested algorithm has the following features. 


1. The initial exclusion from the analysis of pseudo-zero estimates, as some 
students who understand educational material not so good and who are not 
fully involved in the educational process can significantly affect on the overall 
average group assessment of knowledge. At the same time, their level of 
knowledge does not reflect the level of knowledge of students actually 
participating in the educational process. 

2. The asymmetric shift of the estimate relative to the mathematical expectation 
when a subgroup with reduced academic performance is singled out allows 
highlight the core of the group, "suspicious" to an insufficient level of 
knowledge. 

3. Using the modified assessment allows us to take into reasonable limits account 
the average level of academic achievement in the group and stimulate the 
interaction of the students of the group in the studying process. 

4. The suggested algorithm makes it possible to discover the true reasons for the 
lowered academic achievement in the group and organize correction of 
knowledge with minimal additional costs of teaching work. 

5. In addition to all the above, it is necessary to take into account the modular 
estimates in the boundary and final control of knowledge. One of the problems 
is the joint consideration of test results on various topics, for example, 1) ona 
set of modules or 2) on a set of sections in the evaluation of midterm control in 
the final evaluation. Due to the different significance of the individual modules 
in the acquisition of competencies in the studied discipline, it is suggested to 
use normalized weight coefficients and appropriate summation formulas for it. 


At the same time of testing a set of p modules (for example, with a boundary 
knowledge control), they are treated with their partial weights Wi, W2,...Wp. 
Suppose, as a result of testing, were got evaluations of B), B2,...Bp. As a general, 
the following estimate is taken: 


B=(W, Bi + ...+ W,B,)/( Wi + ...+ Wp). 
(6) 

When taking into account the results of the boundary control Br, Br2,...Br; on 
the final attestation of knowledge, these evaluations are also considered with their 
partial weights Wn, Wr2,..., Wrx.. The generalized estimate of the boundary control 
in the final evaluation is the following: 

Br =(Wr, Bri + ...+ WrBre)/( Wri + ...+ Wr). 

(7) 

If the partial weights Wr1, Wrz,..., Wr, are normalized, the denominator in the 

formula 7 is equal to 1. 


CONCLUSION 


It was held the experimental commission of of the suggested method of adaptive 
computer testing in one of the Russian universities, which showed an increase in 
the motivation of the training activities of students of the experimental group, 
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improving the quality of acquired knowledge in the discipline "Programming", 
increasing the desire to cooperate in the group, mastering various aspects of 
professional competencies. 


The positive opinions were received not only from students of experimental 
groups, but also from teachers who taught students in subsequent related 
disciplines, who observed a higher level of knowledge among undergraduate 
students who mastered the course on this method, and also noted an increase in 
the overall positive working mood in the study course of discipline on the 
developed methodology in the experimental groups, which was expressed in the 
desire of students to assimilate the educational material of the course in the 
framework of training sessions, greater concentration and accuracy in carrying 
out tasks of laboratory and test tasks. 


Thus, the results of the pedagogical experiment confirmed the effectiveness 
of the developed method of adaptive computer testing to improve the quality of 
professional education of undergraduate students. 


Till correction (the 
number of persons) 


@ After correction 
(the number of 
persons) 


Oo N FF DD OW 


Diagram I. Change in the number of students failing by sections of the 
course in the experimental group 


The method of adaptive computer testing offered by the authors gives an 
opportunity to take a fresh look at the problem of designing educational 
disciplines in the format of interactive learning, to adapt the process of computer 
testing of students’ knowledge simultaneously by several parameters and to 
achieve higher educational indicators when teaching students at the universities, 
to build an individual trajectory of training. 
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